Dictionary Preconditioning for Greedy Algorithms

Greedy algorithms for Sparse Dictionary Learning

Background. Sparse dictionary learning is a kind of representation learning where we express the data as a sparse linear combination of an overcomplete basis set. This is usually formulated as an optimization problem which is known to be NP-Hard. A typical solution uses a two-step iterative procedure which involves either a convex relaxation or some clustering based solution. One problem with t...

Greedy Deep Dictionary Learning

—In this work we propose a new deep learning tool – deep dictionary learning. Multi-level dictionaries are learnt in a greedy fashion – one layer at a time. This requires solving a simple (shallow) dictionary learning problem; the solution to this is well known. We apply the proposed technique on some benchmark deep learning datasets. We compare our results with other deep learning tools like s...

Greedy algorithms for prediction

In many prediction problems, it is not uncommon that the number of variables used to construct a forecast is of the same order of magnitude as the sample size, if not larger. We then face the problem of constructing a prediction in the presence of potentially large estimation error. Control of the estimation error is either achieved by selecting variables or combining all the variables in some ...

Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection

We study the problem of selecting a subset of k random variables from a large set, in order to obtain the best linear prediction of another variable of interest. This problem can be viewed in the context of both feature selection and sparse approximation. We analyze the performance of widely used greedy heuristics, using insights from the maximization of submodular functions and spectral analys...

Greedy Algorithms